Analysis

Cleaning scripts:

Load data:

Compare probability ratings

Exclude random responses

exclude_random = function(d) {
  d_overall_means = d %>%
  group_by(modal, workerid) %>% 
  summarise(rating_m_overall = mean(rating))

  d_indiv_means =  d %>%
    group_by(modal,percent_window, workerid) %>% 
    summarise(rating_m = mean(rating))
  
  d_indiv_merged = merge(d_indiv_means, d_overall_means, by=c("workerid", "modal"))
  
  cors = d_indiv_merged %>%
    group_by(workerid) %>%
    summarise(corr = cor(rating_m, rating_m_overall))
  
  exclude = cors %>%
    filter(corr > 0.75) %>%
    .$workerid
  
  print(paste("Excluded", length(exclude), "participants based on random responses."))
  
  d = d %>% filter(!(workerid %in% exclude))
}

d1 = exclude_random(d1)

## [1] "Excluded 2 participants based on random responses."

d2 = exclude_random(d2)

## [1] "Excluded 1 participants based on random responses."

d3 = exclude_random(d3)

## [1] "Excluded 1 participants based on random responses."

d4 = exclude_random(d4)

## [1] "Excluded 4 participants based on random responses."

Aggregated results

## Individual plots

plot(ps1$by_participant)

plot(ps2$by_participant)

plot(ps3$by_participant)

plot(ps4$by_participant)

AUC Computation

So, we’re seeing what we expected for the confident speaker, with the AUC > in the optimistic condition than in the confident condition (indicating explaining away), but we’re not seeing the same (with just one speaker) for the pessimistic/cautious condition, where we would expect flipped results, with the pessimistic condition having a lower AUC than the cautious condition (with adaptation). To sum up, we expect the order to be, from greater to lower difference, “cautious”, “pessimistic”, “optimistic”, and “confident”.

## 
##  Two Sample t-test
## 
## data:  aucs.optimist$auc_diff and aucs.confident$auc_diff
## t = 0.72495, df = 13, p-value = 0.4813
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -12.60271  25.33260
## sample estimates:
## mean of x mean of y 
##  9.637793  3.272847

## 
##  Two Sample t-test
## 
## data:  aucs.pessimist$auc_diff and aucs.cautious$auc_diff
## t = -1.1175, df = 12, p-value = 0.2857
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -52.78653  16.99633
## sample estimates:
## mean of x mean of y 
## -2.021748 15.873352